Handling Class Imbalance Problem Using Feature Selection

1 Introduction The class imbalance problem is a challenge to machine learning and data mining, and it has attracted significant research recent years. A classifier affected by the class imbalance problem for a specific data set would see strong accuracy overall but very poor performance on the minority class. The imbalance data sets are pervasive in real-world applications. Examples of these kinds of applications include biological data analysis, text classification, and image classification, web page classification among many others. The skew of an imbalanced data set can be severe. Some imbalanced data sets will only have one minority sample for every 100 majority samples. Researchers have crafted many techniques to combat the class imbalance problem, including resampling, new algorithms, and feature selection. With imbalanced data, classification rules that predict the small classes tend to be fewer and weaker than those that predict the prevalent classes; consequently, test samples belonging to the small classes are misclassified more often than those belonging to the prevalent classes. Standard classifiers usually perform poorly on imbalanced data sets because they are designed to generalize from training data and output the simplest hypothesis that best fits the data. Therefore, the simplest hypothesis pays less attention to rare cases. However, in many cases, identifying rare objects is of crucial importance; classification performances on the small classes are the main concerns in determining the property of a classification model. Why is the class imbalance problem so prevalent and difficult to overcome? First, modern classifiers assume that unseen data points on which the classifier will be asked to make a prediction are drawn from the same distribution as the training data. If testing and validation data samples were drawn from a different distribution, the trained classifier may give poor results because of the flawed model. Based on this assumption, a classifier will almost always produce poor accuracy on an imbalanced data set. In a bi-class application, the imbalanced problem is observed as one class is represented by a large amount of samples while the other is represented by only a few. The class with very few training samples and usually associated with high identification importance, is referred as the positive class; the other one as the negative class. The learning objective of this kind of data is to obtain a satisfactory identification performance on the positive (small) class. Researchers have crafted many techniques to combat the class imbalance problem, …